26th Annual Computational Neuroscience Meeting (CNS*2017): Part 3 - Meeting Abstracts - Antwerp, Belgium. 15–20 July 2017

This work was produced as part of the activities of FAPESP Research,\ud Disseminations and Innovation Center for Neuromathematics (grant\ud 2013/07699-0, S. Paulo Research Foundation). NLK is supported by a\ud FAPESP postdoctoral fellowship (grant 2016/03855-5). ACR is partially\ud supported by a CNPq fellowship (grant 306251/2014-0)

Using a simple neural network to delineate some principles of distributed economic choice

The brain uses a mixture of distributed and modular organization to perform computations and generate appropriate actions. While the principles under which the brain might perform computations using modular systems have been more amenable to modeling, the principles by which the brain might make choices using distributed principles have not been explored. Our goal in this perspective is to delineate some of those distributed principles using a neural network method and use its results as a lens through which to reconsider some previously published neurophysiological data. To allow for direct comparison with our own data, we trained the neural network to perform binary risky choices. We find that value correlates are ubiquitous and are always accompanied by non-value information, including spatial information (i.e., no pure value signals). Evaluation, comparison, and selection were not distinct processes; indeed, value signals even in the earliest stages contributed directly, albeit weakly, to action selection. There was no place, other than at the level of action selection, at which dimensions were fully integrated. No units were specialized for specific offers; rather, all units encoded the values of both offers in an anti-correlated format, thus contributing to comparison. Individual network layers corresponded to stages in a continuous rotation from input to output space rather than to functionally distinct modules. While our network is likely to not be a direct reflection of brain processes, we propose that these principles should serve as hypotheses to be tested and evaluated for future studies.This work was supported by an R01 from NIH to BH (DA037229) and grants PSI2013-44811-P and FLAGERA-PCIN-2015-162-C02-02 from MINECO (Spain) to RM-B

A basal ganglia model of freezing of gait in Parkinson's disease

Freezing of gait (FOG) is a mysterious clinical phenomenon seen in Parkinson’s disease (PD) patients, a neurodegenerative disorder of the basal ganglia (BG), where there is cessation of locomotion under specific contexts. These contexts could include motor initiation, i.e., when starting movement, passing through narrow passages and corridors, while making a turn and as they are about to reach a destination. We have developed computational models of the BG which explains the freezing behaviour seen in PD. The model uses reinforcement learning framework, incorporating Actor-Critic architecture, to aid learning of a virtual subject to navigate through these specific contexts. The model captures the velocity changes (slowing down) seen in PD freezers upon encountering a doorway, turns, and under the influence of cognitive load compared to PD non-freezers and health controls. The model throws interesting predictions about the pathology of freezing suggesting that dopamine, a key neurochemical deficient in PD, might not be the only reason for the occurrences of such freeze episodes. Other neuromodulators which are involved in action exploration and risk sensitivity influence these motor arrests. Finally, we have incorporated a network model of the BG to understand the network level parameters which influence contextual motor freezing

Modeling sertonin's contributions to basal ganglia dynamics in Parkinson's disease with impulse control disorders

Impulsivity involves irresistibility in execution of actions and is prominent in medication condition of Parkinson’s disease (PD) patients. In this chapter, we model a probabilistic reversal learning task in PD patients with and without impulse control disorder (ICD) to understand the basis of their neural circuitry responsible for displaying ICD in PD condition. The proposed model is of the basal ganglia (BG) action selection dynamics, and it predicts the dysfunction of both dopaminergic (DA) and serotonergic (5HT) neuromodulator systems to account for the experimental results. Furthermore, the BG is modelled after utility function framework with DA controlling reward prediction and 5HT controlling the loss and risk prediction, respectively. The striatal model has three pools of medium spiny neurons (MSNs) including those with D1 receptor ® alone, D2R alone, and co-expressing D1R-D2R neurons. Some significant results modelled are increased reward sensitivity during ON medication and an increased punishment sensitivity during OFF medication in patients. The lower reaction times (RT) in ICD subjects compared to that of the non-ICD category of the PD ON patients are also explained. Other modelling predictions include a significant decrease in the sensitivity to loss and risk in the ICD patients

An extended reinforcement learning model of basal ganglia to understand the contributions of serotonin and dopamine in risk-based decision making, reward prediction, and punishment learning

Although empirical and neural studies show that serotonin (5HT) plays many functional roles in the brain, prior computational models mostly focus on its role in behavioral inhibition. In this study, we present a model of risk based decision making in a modified Reinforcement Learning (RL)-framework. The model depicts the roles of dopamine (DA) and serotonin (5HT) in Basal Ganglia (BG). In this model, the DA signal is represented by the temporal difference error (δ), while the 5HT signal is represented by a parameter (α) that controls risk prediction error. This formulation that accommodates both 5HT and DA reconciles some of the diverse roles of 5HT particularly in connection with the BG system. We apply the model to different experimental paradigms used to study the role of 5HT: (1) Risk-sensitive decision making, where 5HT controls risk assessment, (2) Temporal reward prediction, where 5HT controls time-scale of reward prediction, and (3) Reward/Punishment sensitivity, in which the punishment prediction error depends on 5HT levels. Thus the proposed integrated RL model reconciles several existing theories of 5HT and DA in the BG

The motor, cognitive, affective and autonomic functions of the basal ganglia

The basal ganglia are involved in several processes, ranging from motor to cognitive ones. This chapter briefly discusses the role of the basal ganglia in motor (including reaching, handwriting, precision grip, gait, saccade generation, and speech), cognitive (action selection, decision making, attention, working memory, sequence learning, and sleep regulation), mood/emotion (negative and positive affect), and autonomic (gastrointestinal and cardiovascular) processes. The chapter summarizes key experimental studies explaining the role of the basal ganglia in all of these motor, cognitive, and affective processes. Accordingly, this chapter provides a background on the function of the basal ganglia, which is key information that guides the reader to understand the following computational modelling efforts to understand the role of the basal ganglia in several functional processes

Modeling serotonin's contributions to basal ganglia dynamics

In addition to dopaminergic input, serotonergic (5-HT) fibers also widely arborize through the basal ganglia circuits and strongly control their dynamics. Although empirical studies show that 5-HT plays many functional roles in risk-based decision making, reward, and punishment learning, prior computational models mostly focus on its role in behavioural inhibition or timescale of prediction. This chapter presents an extended reinforcement learning (RL)-based model of DA and 5-HT function in the BG, which reconciles some of the diverse roles of 5-HT. The model uses the concept of utility function-a weighted sum of the traditional value function expressing the expected sum of the rewards, and a risk function expressing the variance observed in reward outcomes. Serotonin is represented by a weight parameter, used in this combination of value and risk functions, while the neuromodulator dopamine (DA) is represented as reward prediction error as in the classical models. Consistent with this abstract model, a network model is also presented in which medium spiny neurons (MSN) co-expressing both D1 and D2 receptors (D1R-D2R) is suggested to compute risk, while those expressing only D1 receptors ae suggested to compute value. This BG model includes nuclei such as striatum, Globus Pallidus externa, Globus Pallidus interna, and subthalamic nuclei. DA and 5-HT are modelled to affect both the direct pathway (DP) and the indirect pathway (IP) composing of D1R, D2R, D1R-D2R projections differentially. Both abstract and network models are applied to data from different experimental paradigms used to study the role of 5-HT: (1) risk-sensitive decision making, where 5-HT controls the risk sensitivity; (2) temporal reward prediction, where 5-HT controls timescale of reward predition, and (3) reward-punishment sensitivity, where punishment prediction error depends on 5-HT levels. Both the extended RL model (Balasubramani, Chakravarthy, Ravindran, & Moustafa, in Front Comput Neurosci 8:47, 2014; Balasubramani, Ravindran, & Chakravarthy, in Understanding the role of serotonin in basal ganglia through a unified model, 2012) along with their network correlates (Balasubramani, Chakravarthy, Ravindran, & Moustafa, in Front Comput Neurosci 9:76, 2015; Balasubramani, Chakravarthy, Ali, Ravindran, & Moustafa, in PLoS ONE 10(6):e0127542, 2015) successfully explain the three diverse roles of 5-HT in a single framework

Image2.PNG

<p>The brain uses a mixture of distributed and modular organization to perform computations and generate appropriate actions. While the principles under which the brain might perform computations using modular systems have been more amenable to modeling, the principles by which the brain might make choices using distributed principles have not been explored. Our goal in this perspective is to delineate some of those distributed principles using a neural network method and use its results as a lens through which to reconsider some previously published neurophysiological data. To allow for direct comparison with our own data, we trained the neural network to perform binary risky choices. We find that value correlates are ubiquitous and are always accompanied by non-value information, including spatial information (i.e., no pure value signals). Evaluation, comparison, and selection were not distinct processes; indeed, value signals even in the earliest stages contributed directly, albeit weakly, to action selection. There was no place, other than at the level of action selection, at which dimensions were fully integrated. No units were specialized for specific offers; rather, all units encoded the values of both offers in an anti-correlated format, thus contributing to comparison. Individual network layers corresponded to stages in a continuous rotation from input to output space rather than to functionally distinct modules. While our network is likely to not be a direct reflection of brain processes, we propose that these principles should serve as hypotheses to be tested and evaluated for future studies.</p

Image3.PNG

<p>The brain uses a mixture of distributed and modular organization to perform computations and generate appropriate actions. While the principles under which the brain might perform computations using modular systems have been more amenable to modeling, the principles by which the brain might make choices using distributed principles have not been explored. Our goal in this perspective is to delineate some of those distributed principles using a neural network method and use its results as a lens through which to reconsider some previously published neurophysiological data. To allow for direct comparison with our own data, we trained the neural network to perform binary risky choices. We find that value correlates are ubiquitous and are always accompanied by non-value information, including spatial information (i.e., no pure value signals). Evaluation, comparison, and selection were not distinct processes; indeed, value signals even in the earliest stages contributed directly, albeit weakly, to action selection. There was no place, other than at the level of action selection, at which dimensions were fully integrated. No units were specialized for specific offers; rather, all units encoded the values of both offers in an anti-correlated format, thus contributing to comparison. Individual network layers corresponded to stages in a continuous rotation from input to output space rather than to functionally distinct modules. While our network is likely to not be a direct reflection of brain processes, we propose that these principles should serve as hypotheses to be tested and evaluated for future studies.</p

The many facets of dopamine : toward an integrative theory of the role of dopamine in managing the body's energy resources

In neuroscience literature, dopamine is often considered as a pleasure chemical of the brain. Dopaminergic neurons respond to rewarding stimuli which include primary rewards like opioids or food, or more abstract forms of reward like cash rewards or pictures of pretty faces. It is this reward-related aspect of dopamine, particularly its association with reward prediction error, that is highlighted by a large class of computational models of dopamine signaling. Dopamine is also a neuromodulator, controlling synaptic plasticity in several cortical and subcortical areas. But dopamine's influence is not limited to the nervous system; its effects are also found in other physiological systems, particularly the circulatory system. Importantly, dopamine agonists have been used as a drug to control blood pressure. Is there a theoretical, conceptual connection that reconciles dopamine's effects in the nervous system with those in the circulatory system? This perspective article integrates the diverse physiological roles of dopamine and provides a simple theoretical framework arguing that its reward related function regulates the processes of energy consumption and acquisition in the body. We conclude by suggesting that energy-related book-keeping of the body at the physiological level is the common motif that links the many facets of dopamine and its functions